Discretization-aware architecture search

The search cost of neural architecture (NAS) has been largely reduced by differentiable and weight-sharing methods. Such methods optimize a super-network with all possible edges operations, determine the optimal sub-network discretization, i.e., pruning off operations/edges small weights. However, discretization process performed on either operations or incurs significant inaccuracy thus quality is not guaranteed. In this paper, we propose discretization-aware (DA2S), target at pushing towards configuration desired topology. DA2S implemented an entropy-based loss term, which can be regularized to in plug-and-play fashion. regularization controlled elaborated continuation functions, so that adaptive dynamic change operations. Experiments standard image classification benchmarks demonstrate effectiveness our approach, particular, under imbalanced network configurations were studied before. Code available github.com/sunsmarterjie/DAAS.